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United  States  General  Accounting  Office 

COAST  GUARD 
Update  on  Marine  Information  for  Safety 
and  Law  Enforcement  System 


Highlights  of  GAO-02-11,  a  report  to  the  Subcommittee  on  Coast  Guard  and  Maritime  Transportation, 
House  Committee  on  Transportation  and  Infrastructure 


Why  GAO  Did  This  Study 

The  Coast  Guard  is  developing  a 
Web-based  information  system  to 
replace  an  aging  computer 
system  that  it  uses  to  track  safety 
and  law-enforcement  actions — 
such  as  inspections,  drug 
interdiction,  and  oil  spill 
assistance — involving 
commercial  and  recreational 
vessels.  In  1995,  the  Coast  Guard 
awarded  a  contract  to  develop 
the  new  system,  called  MISLE. 
After  the  project  encountered 
cost  and  schedule  problems, 
development  responsibility  was 
transferred  to  the  Coast  Guard’s 
systems  development  center  in 
1999,  and  the  contract  terminated 
after  about  $26  million  had  been 
spent. 

The  Coast  Guard’s  history  of 
systems  development  problems 
and  information  technology 
weaknesses  prompted  the 
Subcommittee  to  ask  GAO  to 
review  MlSLE’s  current  status 
and  risks. 


What  GAO  Recommends 

GAO  is  making  several 
recommendations  in  the  four  key 
areas  of  system  requirements, 
software  testing,  transition 
plaiming,  and  risk  mitigation. 

In  comments  on  a  draft  of  our 
report.  Coast  Guard  officials 
generally  agreed  with  GAO’s 
recommendations,  but  noted  that 
some  of  the  risks  were  “tradeoff 
decisions,”  necessary  to  deliver  an 
initial  version  of  MISLE. 


What  GAO  Found 

The  Coast  Guard  has  made  progress  since  taking  over  its  Marine 
Information  for  Safety  and  Law  Enforcement  (MISLE)  system 
development;  it  is  now  poised  to  deploy  the  system  with  hmited 
capabilities.  But  much  remains  to  be  accomplished  in  order  to  deliver  the 
complete  system,  with  only  about  $11  million  of  its  $61  million  estimate 
remaining. 

MISLE  is  facing  risks  in  four  key  areas: 

•  Changing  system  requirements.  MISLE  requirements  have 
repeatedly  changed;  several  have  been  dropped,  and  others 
postponed  to  later  development  phases  due  to  problems  found  during 
testing  and  the  Coast  Guard’s  emphasis  on  replacing  its  current 
system  as  soon  as  possible.  Other  requirements  are  being  added  or 
accelerated.  Such  a  continually  changing  scenario  increases  the  risk 
that  MISLE  will  fall  short  of  user  needs. 

•  Software  testing.  The  Coast  Guard  imdertook  risky  software 
testing  practices  in  that  it  deferred  testing  some  functions  and  did  not 
resolve  all  critical  problems  imcovered  before  moving  on  to  the  next 
testing  stage.  This  approach  increases  the  likelihood  that  the  system 
will  not  perform  as  expected  and/or  may  take  longer  to  develop  than 
anticipated. 

•  Transition  planning.  Deployment  of  MISLE  involves  planning  for 
accurately  moving  data  from  the  older  system  and  training  system 
users.  However,  critical  transition  plans  are  not  yet  complete. 

Beyond  adding  to  possible  delays,  the  absence  of  transition  plans  and 
insufficient  training  increase  the  chance  of  user  discomfort  with  the 
new  system. 

•  Risk  management.  Finally,  the  Coast  Guard’s  risk  management 
approach  has  been  ineffective:  risks  were  not  assigned  severity 
ratings,  and  not  all  have  been  prioritized.  Further,  the  Coast  Guard 
has  not  developed  detailed  mitigation  plans  for  all  significant  risks. 

Unless  these  challenges  are  successfully  addressed,  performance 
shortcomings,  cost  escalation,  and  schedule  delay  are  likely. 


This  is  a  test  for  developing  Highlights  for  a  GAO  report.  For  additional  information  abont  the  report,  call  Linda  Koontz  at  (202)  512- 
6240.  To  provide  comments  on  this  test  Highlights  page,  call  Keith  Fnltz  at  (202)  512-3200  or  send  an  email  to  HighlightsTest@gao.gov. 
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United  States  General  Accounting  Office 
Washington,  D.C.  20548 


October  17,  2001 

The  Honorable  Frank  LoBiondo 
Chairman 

The  Honorable  Corrine  Brown 
Ranking  Minority  Member 

Subcommittee  on  Coast  Guard  and  Maritime  Transportation 
Committee  on  Transportation  and  Infrastructure 
House  of  Representatives 

Over  the  last  decade,  the  United  States  Coast  Guard  (USCG)  has 
experienced  difficulties  in  acquiring  the  Marine  Information  for  Safety  and 
Law  Enforcement  (MISLE)  system — an  information  system  to  track  marine 
safety  and  law-enforcement  activities  involving  commercial  and 
recreational  vessels.  In  1999,  the  Coast  Guard  terminated  a  contract  to 
acquire  MISLE,  after  spending  about  4  years  and  $26  million,  and  is  instead 
developing  the  system  at  its  Operations  Systems  Center.  As  you  requested, 
our  objective  was  to  provide  an  update  on  MISLE’s  status,  plans,  and 
technical  and  programmatic  risks. 

To  fulfill  this  objective,  we  evaluated  MISLE  project  plans,  costs,  and 
schedules  by  comparing  original  and  current  job  management  documents. 
We  also  assessed  technical  and  programmatic  risks  facing  the  MISLE 
acquisition  and  USCG’s  plans  for  addressing  those  risks.  We  performed  our 
work  from  May  through  September  2001,  in  accordance  with  generally 
accepted  government  auditing  standards.  USCG  officials  provided  us  with 
comments  on  a  draft  of  this  report;  they  are  discussed  in  the  “Agency 
Comments”  section. 

On  September  18,  2001,  we  provided  a  detailed  briefing  to  your  office  on 
the  results  of  this  work.  The  briefing  slides  are  included  in  appendix  1.  The 
purpose  of  this  letter  is  to  provide  the  published  briefing  slides  to  you  and 
to  officially  transmit  our  recommendations  to  the  Secretary  of 
Transportation. 

In  brief,  we  reported  that  USCG  has  made  progress  in  developing  MISLE 
and  was  poised  to  deploy  a  minimum  level  of  functionality  in  November 
2001.  However,  the  system  was  several  months  behind  schedule,  and 
USCG  had  already  spent  most  of  the  $61  million  acquisition  cost  estimate — 
most  of  it  on  the  original  contract.  With  only  about  $11  million  remaining, 
much  remains  to  be  done  to  deliver  the  complete  system.  In  its  efforts  to 
develop  and  deploy  a  complete  MISLE  system,  USCG  faces  significant 
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challenges  and  risks  in  several  areas,  including  managing  system 
requirements  and  user  expectations,  testing  the  system,  transitioning  to  an 
operational  system,  and  managing  program  risks.  We  made  specific 
recommendations  to  address  these  risks. 


RGCOrmTlGridcLtionS  mitigate  USCG’s  MISLE  risks,  we  recommend  that  the  Secretary  of 

Transportation  direct  the  USCG  Commandant  to  ensure  that  the 

appropriate  officials  complete  the  following  actions. 

In  the  system  requirements  area, 

•  define  and  prioritize,  in  conjunction  with  system  users,  all  needed 
system  functions,  corrections,  and  enhancements  that  must  occur  to 
meet  valid  user  needs;  and 

•  develop  cost  and  schedule  estimates  for  providing  these  functions, 
corrections,  and  enhancements. 

In  the  software  testing  area, 

•  close  all  critical  problems  before  initiating  the  next  state  of  testing. 

In  the  area  of  transition  planning, 

•  finalize  and  implement  Vessel  Documentation  System  (VDS)  transition 
plans,  and 

•  develop  and  implement  VDS  training  materials. 

In  the  risk  mitigation  area, 

•  develop  a  single  list  of  system  risks, 

•  evaluate  system  risks  to  determine  their  severity  and  prioritize  these 
risks, 

•  develop  and  implement  comprehensive  mitigation  strategies  for  each  of 
the  risks,  and 

•  regularly  oversee  the  status  of  risks  and  risk  mitigation  efforts  to 
determine  whether  additional  mitigation  activities  are  warranted. 
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Agency  Comments 


We  provided  drafts  of  our  briefing  and  this  report  to  Department  of 
Transportation  and  USCG  officials,  including  representatives  of  the  Office 
of  the  Secretary  of  Transportation  and  the  USCG  Acquisition  Management 
Office.  We  met  with  USCG  officials  to  obtain  their  comments  on  our  drafts. 
These  officials  generally  agreed  with  our  recommendations,  but 
characterized  the  changing  requirements  and  testing  issues  we  raised  as 
“tradeoff  decisions,”  necessary  to  deliver  an  initial  version  of  MISLE. 
USCG  officials  also  noted  that,  given  the  recent  terrorist  attacks,  MISLE 
deployment  will  likely  be  delayed  until  at  least  December  2001. 


We  are  sending  copies  of  this  report  to  the  Secretary  of  Transportation,  the 
USCG  Commandant,  the  Director  of  the  Office  of  Management  and  Budget, 
and  other  interested  parties.  Copies  will  also  be  made  available  to  others 
upon  request. 

Should  you  or  your  staffs  have  any  questions  concerning  this  report,  please 
contact  me  at  (202)  512-6240  or  by  e-mail  at  koontzl@gao.gov.  Nabajyoti 
Barkakati,  Barbara  Collier,  Michael  Fruitman,  Colleen  Phillips,  Margaret 
Sullivan,  and  Glenda  Wright  were  major  contributors  to  this  report. 


Linda  D.  Koontz 

Director,  Information  Management  Issues 
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Appendix  I 


GAO’s  September  18,  2001,  Briefing 


GAO 

Accountability  *  Integrity  *  Reliability 


United  States  Coast  Guard’s  (USCG) 

Marine  Information  for  Safety  and  Law  Enforcement  (MISLE)  System 

An  Update 


Briefing  for  Staff 

Subcommittee  on  Coast  Guard  and  Maritime  Transportation 
Committee  on  Transportation  and  Infrastructure 
House  of  Representatives 

September  18,  2001 
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Appendix  I 

GAO’s  September  18,  2001,  Briefing 


A  G  A  O 

fc  VJ  -n.  W  Briefing  Outiine 

Accountability  *  Integrity  *  Reliability _ ^ _ 

•  Objective,  Scope,  and  Methodology 

•  MISLE  Background 

•  MISLE  Status  and  Plans 

•  Key  Risk  Areas 

•  System  requirements 

•  Software  testing 

•  Transition  planning 

•  Risk  management 

•  Conclusions 

•  Recommendations 

•  Agency  Comments  and  Our  Evaluation 

2 


Page  5 


GAO-02-11  Coast  Guard 


Appendix  I 

GAO’s  September  18,  2001,  Briefing 
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Objective,  Scope,  and  Methodology 


Objective 

To  provide  an  update  on  MISLE’s  status,  plans,  and  technical  and 
programmatic  risks 


Scope  and  Methodology 

We  evaluated  MISLE  project  plans,  costs,  and  schedules  by  comparing 
original  and  current  job  management  documents.  We  also  assessed  technical 
and  programmatic  risks  facing  the  MISLE  acquisition,  as  well  as  USCG’s 
plans  for  addressing  those  risks,  by  evaluating  USCG  program  documents, 
comparing  them  to  accepted  system  engineering  principles,  and  interviewing 
project  officials,  developers,  and  system  users.  Further,  we  reviewed  plans 
for  future  MISLE  enhancements  and  discussed  these  enhancements  with 
system  users  and  developers. 

We  conducted  our  review  at  USCG  headquarters  in  Washington,  D.C.,  and  at 
USCG’s  Operations  Systems  Center  and  National  Vessel  Documentation 
Center  in  the  Martinsburg,  WV,  area.  We  conducted  our  work  from  May 
through  September  2001 ,  in  accordance  with  generally  accepted  government 
auditing  standards. 
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GAO’s  September  18,  2001,  Briefing 
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MISLE  Background 

Overview 


MISLE  is  intended  to  be  a  Web-based  information  system  for  entering  and 
obtaining  data  on  Coast  Guard  activities  concerning  commercial  and 
federally  documented  recreational  vessels  in  support  of  USCG’s  marine 
safety  and  law  enforcement  missions. 


A  software-intensive  system,  the  MISLE  project  also  includes  the  purchase 
of  hardware,  including  network  servers  and  data  storage  devices. 

MISLE  is  to  support  over  5000  system  users  at  about  550  sites  using  the 
existing  Coast  Guard  intranet. 
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GAO’s  September  18,  2001,  Briefing 
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MISLE  Background 

Overview  (cont’d) 


In  the  mid-1990s,  MISLE  was  envisioned  to  develop  and  integrate  two  major 
systems — the  Marine  Safety  Network  and  the  Vessel  Identification  and 
Documentation  System — and  to  integrate  them  with  a  third,  already  developed 
system,  the  Law  Enforcement  Information  System  II. 


Since  then,  however,  this  concept  has  evolved.  A  description  of  the 
subsystems  and  recent  changes  to  these  subsystems  are  discussed  below: 

•  Marine  Safety  Network  (MSN) 

a  system  to  allow  Coast  Guard  personnel  to  input  and  obtain  information 
on  USCG  marine  safety  activities,  such  as  vessel  inspections  and 
boardings;  it  relies  on  vessel  data  from  the  Vessel  Documentation  System 
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GAO’s  September  18,  2001,  Briefing 
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MISLE  Background 

Overview  (cont’d) 


MISLE  components  (cont’d) 


•  Vessel  Identification  and  Documentation  System 

a  system  planned  to  integrate  two  subsystems  in  response  to  Public  Law 
100-710  (commonly  called  the  Ship  Mortgage  Act  of  1988): 

•  Vessel  Identification  System — planned  to  automate  and  integrate 
vessel  registration  information  from  participating  states;  due  to 
complications  in  implementing  this  system,  USCG  no  longer  plans  to 
provide  VIS  functionality  as  part  of  MISLE  (see  Key  Risk  Area — System 
Requirements) 

•  Vessel  Documentation  System  (VDS) — a  system  to  support  the 
National  Vessel  Documentation  Center’s  processes  for  documenting 
vessels 


•  Law  Enforcement  Information  System  II  (LEIS-II) 

a  system  for  tracking  USCG  law  enforcement  activities,  such  as  drug 
interdiction  activities;  due  to  the  age  of  the  LEIS-II  platform,  USCG  now 
plans  to  replace  LEIS-II  rather  than  integrate  it  with  MISLE 
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GAO’s  September  18,  2001,  Briefing 
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MISLE  Background 

Overview  (cont’d) 


As  of  early  September  2001 ,  USCG’s  estimates  for  MISLE  costs 
and  deployment  schedules  included  the  following: 


•  Acquisition  cost:  $61  million  through  2003 

•  Life  cycle  cost:  $94  million  through  2008 

•  Initial  capability:  November  2001 

•  Final  capability:  September  2003 
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GAO’s  September  18,  2001,  Briefing 
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MISLE  Background 

History _ 


In  1986,  USCG  began  developing  requirements  for  replacing  its 
legacy  Marine  Safety  Information  System  (MSIS)  by  1995. 


In  the  early  1990s,  USCG  delayed  plans  to  replace  MSIS  in  order  to 
integrate  requirements  for  multiple  systems  into  one  system 
development  effort — called  MISLE. 


In  1995,  USCG  awarded  a  contract  to  Computer  Sciences 
Corporation  (CSC)  to  develop  and  deliver  a  complete  MISLE  system 
by  2002.  At  that  time,  USCG  officials  estimated  that  this  contract 
could  cost  up  to  $35  million. 


•  A  March  1999  change  required  the  developer  to  replace  MSIS 
functions  by  June  2001 . 


8 


Page  11 


GAO-02-11  Coast  Guard 


Appendix  I 

GAO’s  September  18,  2001,  Briefing 
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MISLE  Background 

History  (cont’d) _ 


The  MISLE  development  contract  encountered  escalating  costs  and 
schedule  delays;  it  was  “partially  terminated  for  government  convenience” 
in  October  1999. 


•  CSC  delivered  two  components  under  this  contract: 

•  Vessel  Identification  System — a  system  to  automate  and 
integrate  vessel  registration  information  from  participating  states. 

This  system  was  never  implemented  due  to  problems  in 
integrating  state  data.  USCG  no  longer  plans  to  provide  VIS 
functionality  as  part  of  MISLE. 

•  Mission  Analysis  and  Planning — a  decision  support  tool  for 
obtaining  statistical  information  from  the  legacy  MSIS  system 

•  Total  spent  on  the  CSC  contract:  $26  million 

In  October  1999,  USCG  transferred  responsibility  for  MISLE  development 
from  CSC  to  its  Operations  Systems  Center  (OSC) — a  government-owned, 
contractor-operated  facility.  USCG’s  acquisition  office  manages  the 

project. _ 
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GAO’s  September  18,  2001,  Briefing 
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MISLE  Background 

Acquisition  Strategy 


OSC’s  strategy  is  to  develop  and  deploy  a  minimal  level  of  MISLE 
functionality  as  soon  as  possible  to  replace  MSIS. 


•  MSIS — once  scheduled  for  termination  in  1995 — is  still  serving  as  the 
primary  information  system  for  marine  safety  programs. 

•  MSIS  has  failed  repeatedly  over  the  past  several  years,  causing  days 
and  weeks  of  data  processing  backlogs. 

•  USCG  officials  stated  that  MSIS  hardware  is  no  longer  supportable; 
the  original  vendor  is  no  longer  in  business,  and  the  agency  is 
currently  reusing  parts  to  keep  the  system  running. 
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GAO’s  September  18,  2001,  Briefing 
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MISLE  Background 

Acquisition  Strategy  (cont’d) 


OSC’s  system  development  approach: 


•  Rapid  Application  Development/Joint  Application  Design 

a  system  development  methodology  that  focuses  on  involving  users 
early  and  often  in  the  design  and  development  of  a  system  in  order  to 
develop  it  quickly. 


•  Timebox  Methodology 

a  system  development  methodology,  often  used  in  conjunction  with 
the  Rapid  Application  Development  methodology,  in  which  a  time 
limit  is  established  for  developing  a  system  (this  forces  users  to  focus 
on  a  minimal  set  of  requirements  and  allows  them  to  drop  some 
functions  if  necessary  to  meet  the  time  limit). 
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MISLE  Background 

Planned  Delivery  Schedule 


OSC’s  three-phased  delivery  approach: 

MISLE  Phase  1 

Oct.  1999  to  Nov.  2001 

Deliverables: 

•  initial  MSN  capability  needed  to  replace  MSIS 

•  initial  VDS  capability 

MISLE  Phase  2 

Feb.  2001  to  March  2002 
Deliverables: 

•  replacement  of  LEIS-II 

•  extended  MSN  and  VDS  capabilities 

•  initial  data  warehouse 


MISLE  Phase  3 

Feb.  2002  to  Sept.  2003 

Deliverables: 

•  completion  of  remaining  MSN  and  VDS  capabilities 
»  expanded  data  warehouse _ 
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MISLE  Status  and  Plans 
Progress  to  Date _ 


MISLE  Phase  1 

40  high  level  requirements  have  been  translated  to  over  13,000  detailed 
software  requirements  supporting  two  key  components:  the  Marine  Safety 
Network  (MSN)  and  the  Vessel  Documentation  System  (VDS) 


MSN  Status: 

•  Completed  system  testing^ 

•  User  input  obtained  throughout 


VDS  Status: 

•  Major  components  postponed  to  later  releases 

•  Completed  system  testing 


MISLE  Phase  2 

Development  began  February  2001 


'USCG  tests  increasingly  larger  portions  ot  MiSLE  in  stages:  (1)  unit  testing-informal  tests  of  software  modules;  (2)  functional  testing-formal  tests  of 
software  modules;  (3)  integration  testing-testing  the  integration  of  all  modules  in  a  core  area;  and  (4)  system  testing-testing  all  of  the  core  areas  in  a 
deliverable.  MISLE  then  undergoes  operational  test  and  evaluation  (OT&E)-testing  of  operational  effectiveness  and  suitability  by  system  users. 
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MISLE  Status  and  Plans 
Costs 


USCG’s  $61  million  total  estimated  MISLE  acquisition  cost  includes: 


•  about  $7  million  spent  on  requirements  analysis,  design,  and 
validation  in  the  early  1990s 

•  $26  million  spent  on  the  1995  MISLE  contract  that  was  terminated 

•  about  $28  million  for  current  MISLE  development  efforts,  including 

•  almost  $17  million  spent  through  fiscal  year  2001  to  develop 

•  most  of  MISLE  Phase  1 

•  part  of  MISLE  Phase  2 

•  about  $1 1  million  in  fiscal  year  2002  and  beyond 

•  to  complete  residual  tasks  supporting  MISLE  Phases  1  and  2 

•  to  develop  and  implement  MISLE  Phase  3 
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MISLE  Status  and  Plans 
Schedule 


MISLE  Phase  1  has  fallen  behind  schedule. 


MISLE  Phase  1 
milestones 

Dec.  1999 

baseline 

change” 

June  and 
Dec.  2000 
estimates'’ 

May  2001 
estimate 

Aug. 1, 2001 
estimate 

Sept.  5,  2001 
estimate 

System  testing  complete 

N/r 

N/l 

7/20/01 

MSN  8/15/01 
YDS  9/15/01 

MSN  8/27/01 
YDS  8/27/01 

Operational  Test  and 
Evaluation  complete 

6/01 

2/01 

8/15/01 

MSN  8/30/01 
YDS  9/30/01 

MSN  9/21/01 
YDS  9/21/01 

MISLE  month  complete 

N/l 

N/l 

9/15/01 

9/30/01 

10/19/01 

Data  migration  complete 

N/l 

N/l 

9/30/01 

10/10/01 

11/2/01 

MISLE  Phase  1  deployed 

N/l 

4/01 

10/01 

10/15/01 

11/5/01 

“In  commenting  on  a  draft  of  this  briefing,  the  MISLE  project  manager  noted  this  baseline  document  also  includes 
an  OT&E  completion  range  from  January  to  September  2001 ,  and  that  OT&E  would  be  completed  within  this 
range.  However,  USCG  internal  management  reports  reflect  the  June  2001  completion  date.  In  fact,  these  reports 
show  MISLE  in  breach  of  its  schedule. 

“These  estimates  are  identified  in  the  June  2000  MISLE  Implementation  Plan,  as  well  as  the  December  2000 
MISLE  Acquisition  and  Project  Management  Plans.  In  commenting  on  a  draft  of  this  briefing,  USCG  officials 
characterized  these  documents  as  internal,  optimistic  project  plans--not  firm  schedule  estimates. 

‘Not  identified. 
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MISLE  Status  and  Plans 
Schedule  (cont’d) _ 


USCG  officials  identified  the  following  reasons  for  schedule  delays: 


•  Problems  in  integrating  VDS  and  MSN  components  caused  a 
major  VDS  redesign  (October  2000) 

•  USCG  needed  to  retrofit  MSN  to  work  with  the  current  version  of 
Internet  Explorer  (January  2001) 

•  VDS  functional  testing  was  delayed  because  of  system  instability 
(January  2001) 

•  MSN  integration  testing  identified  a  missing  function  (June  2001): 

•  ability  to  communicate  with  USCG’s  financial  center 

•  VDS  integration  testing  encountered  problems  (July  2001): 

•  test  scripts  would  not  run  at  users’  facility 

•  shortcoming  in  ability  to  edit  MSIS  data  migrated  to  MISLE 
(estimated  4  week  delay) 
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Key  Risk  Areas 


The  MISLE  acquisition  is  facing  considerable  challenges  in  the  following 
key  risk  areas: 

•  System  Requirements 

•  Software  Testing 

•  Transition  Planning 

•  Risk  Management 
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Key  Risk  Areas 

System  Requirements 


System  requirements  define  the  minimum  functions  and  performance 
levels  needed  to  satisfy  user  needs.  USCG  identified  system 
requirements  for  MISLE  in  1994.  Also,  when  the  agency  restructured  the 
MISLE  program  in  1999,  it  identified  specific  functions  to  be  delivered  in 
each  of  the  three  MISLE  phases. 

However,  MISLE  requirements  and  the  functions  to  be  provided  in  each 
of  the  three  deliverables  are  now  changing.  USCG  plans  to  drop  several 
of  MISLE’s  original  system  requirements  and  has  recently  postponed  key 
functions  due  to  problems  encountered  during  testing  and  the  agency’s 
need  to  field  a  system  quickly.  Also,  in  response  to  system  users’  needs, 
USCG  plans  to  add  and  accelerate  key  functions.  Further,  while  testing 
and  training  on  the  system,  users  identified  new  and  missing  functions 
which  USCG  plans  to  add  to  MISLE  later. 

USCG’s  plans  for  dropping,  delaying,  accelerating,  and  adding  functions 
are  discussed  below. 
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Key  Risk  Areas 

System  Requirements  (cont’d) 


USCG  is  planning  to  drop  significant  MISLE  requirements: 

•  Vessel  Identification  System — This  system,  planned  to  automate 
states’  vessel  registration  data  and  integrate  it  with  data  on  federally 
documented  vessels  in  response  to  the  Ship  Mortgage  Act  of  1988, 
was  developed  as  part  of  MISLE  but  never  successfully 
implemented.  USCG  is  considering  developing  a  new  system  outside 
MISLE. 

•  Portable/Remote  Access — This  function  was  planned  to  provide  the 
ability  to  access  MISLE  information  from  remote  locations,  such  as 
Coast  Guard  cutters.  USCG  officials  stated  that  MISLE  will  allow  for 
remote  access  via  laptop  and  dial-up  modem,  but  that  providing 
cutter  connectivity  will  require  decisions  on  the  USCG  infrastructure 
that  are  beyond  MISLE’s  scope.  USCG  is  considering  leasing 
satellite  time  to  provide  cutter  connectivity. 
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Key  Risk  Areas 

System  Requirements  (cont’d) 


Dropped  requirements  (cont’d) 


•  External  Access — This  function  was  planned  to  provide  non-USCG 
personnel  with  access  to  MISLE  data.  USCG  officials  stated  that  MISLE 
has  this  capability  but  will  not  provide  external  access  because  of 
security  concerns.  USCG  plans  to  reassess  the  need  for  external 
access  on  a  case-by-case  basis. 


•  Merchant  Mariner  Licensing  and  Documentation — This  function  was 
to  provide  licensing  and  documentation  capabilities  within  MISLE. 
However,  instead  of  waiting  for  MISLE,  USCG  decided  to  develop  a 
stand-alone  system  to  provide  this  capability.  MISLE  Phase  1  is 
planned  to  link  to  the  Merchant  Mariner  Licensing  and  Documentation 
stand-alone  system. 
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Key  Risk  Areas 

System  Requirements  (cont’d) 


Dropped  requirements  (cont’d) 


•  Pollution  Funds  Management — This  function  was  to  provide  the 
ability  to  manage  pollution  funds  within  MISLE.  Instead,  USCG 
developed  an  alternative  system  outside  MISLE  to  provide  this 
functionality.  MISLE  will  provide  a  link  to  that  system. 

•  Personnel  Resource  Management — This  function  was  to  provide  the 
ability  to  manage  personnel  within  MISLE.  Instead,  USCG  developed 
an  alternative  system  outside  MISLE  to  provide  this  functionality.  USCG 
officials  expect  that  MISLE  will  be  able  to  access  data  in  that  system. 
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Key  Risk  Areas 

System  Requirements  (cont’d) 


USCG  has  delayed  some  MISLE  functions: 

Significant  Phase  1  deliverables  are  now  planned  for  later  releases. 

Examples  include  VDS’  ability  to 

•  provide  Abstracts  of  Titles 

•  provide  Certificates  of  Ownership 

•  automate  NVDC’s  paper-intensive  work  processes 

Key  Phase  2  deliverables  are  now  planned  to  be  Phase  3  deliverables. 

Two  high  priority  examples  include 

•  VDS’  ability  to  handle  requests  for  nonvessel  information  (this 
involves  non-U. S.  corporations  and  oil  spill  response 
organizations) 

•  MSN’s  ability  to  capture  information  on  the  approval  and  oversight 
of  the  construction  of  portable  quarters  on  vessels 
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Key  Risk  Areas 

System  Requirements  (cont’d) 


Delayed  functions  (cont’d) 


Other  Phase  2  deliverables  are  being  reconsidered — that  is,  USCG 
reported  that  they  may  not  be  “doable”  in  Phase  3.  Examples  include 
MSN’s  ability  to 


•  submit  Web  forms,  such  as  vessel  arrival,  inspection  request,  and 
marine  event  application  forms 

•  capture  and  track  information  on  reviews  of  manuals  (such  as 
Operations  Manuals,  Emergency  Evacuation  Plans,  and 
Passenger  Terminal  Security  Plans) 

Other  examples  include  VDS’  ability  to 

•  provide  electronic  access  to  data  in  response  to  FOIA  requests 

•  allow  e-commerce,  such  as  credit  card  transactions  and 
submission  of  Web  forms 


23 


Page  26 


GAO-02-11  Coast  Guard 


Appendix  I 

GAO’s  September  18,  2001,  Briefing 


i 


GAO 

Accountability  *  Integrity  *  Reliability 


Key  Risk  Areas 

System  Requirements  (cont’d) 


USCG  has  added  and  accelerated  MISLE  functions: 

In  response  to  user  needs,  USCG  has  decided  to 

•  replace  LEIS-II  in  Phase  2,  rather  than  integrate  it  with  MISLE 

•  accelerate  a  critical  function,  incident  response  planning,  from 
Phase  3  to  Phase  2 

In  commenting  on  a  draft  of  this  briefing,  USCG  officials  noted  that  the 
agency  has  also  moved  multiple  functions  from  Phase  2  to  later  releases 
of  Phase  1  in  response  to  user  requests  to  get  these  functions  sooner. 
However,  in  prior  discussions,  USCG  officials  acknowledged  that  they 
have  not  yet  scheduled  these  later  releases  and,  in  fact,  some  of  the  later 
Phase  1  releases  may  not  be  delivered  until  after  Phase  2  is  delivered. 
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Key  Risk  Areas 

System  Requirements  (cont’d) 


USCG  plans  to  add  functions  to  MISLE 


While  testing  and  training  on  MISLE,  system  testers  and  users  are 
identifying  both  needed  system  corrections  and  new  user  needs  that 
MISLE  does  not  currently  provide.  USCG  plans  to  add  these  functions  to 
MISLE  as  “future  enhancements.” 


•  As  of  August,  USCG  identified  72  future  enhancements.  Examples 
include 

•  making  vessel  details  on  maps  readable 

•  incorporating  data  from  additional  information  systems  outside 
MISLE 

•  making  a  print  icon  work 

In  commenting  on  a  draft  of  this  briefing,  MISLE  project  officials  stated 
that  these  future  enhancements  are  low-level  requirements  and  user 
preferences,  and  that  they  do  not  jeopardize  the  project’s  ability  to  deliver 
on  its  agreed-upon  scope. 
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Key  Risk  Areas 

System  Requirements  (cont’d) 


USCG  has  developed  a  schedule  for  the  functions  that  it  deems  are  the 
highest  priority — MSN  release  1.1  and  1.2,  and  MISLE  Phase  2. 
However,  USCG  has  not  yet  prioritized  all  of  the  additional  and  delayed 
MISLE  functions,  nor  has  it  developed  schedules  for  providing  them.  For 
example,  USCG  has  not  yet  established  a  schedule  for  delivering  the 
significant  VDS  functions  that  were  deferred  from  Phase  1  to  the  VDS 
release  1 .1 ,  or  for  the  many  other  MSN  and  VDS  functions  anticipated  in 
later  releases  of  Phase  1 . 


By  dropping  and/or  delaying  key  MISLE  functions,  USCG  runs  the  risk 
that  the  system  will  not  function  as  intended  or  expected,  and  that  the 
deployed  system  will  fall  short  of  user  needs.  Further,  without  a  schedule 
or  cost  estimate  for  the  delayed  functions  and  enhancements,  it  is  not 
clear  when  these  functions  will  be  implemented  nor  how  much  it  will  cost 
to  implement  them. 
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Key  Risk  Areas 

Software  Testing 


According  to  leading  information  technology  organizations,  to  be  effective, 
software  testing  practices  should  be  planned  and  conducted  in  a  structured 
and  disciplined  fashion.  Typically,  this  involves  testing  increasingly  larger 
increments  of  a  system  until  the  complete  system  is  tested  and  accepted, 
and  resolving  critical  problems  before  moving  to  the  next  phase  of  testing. 


USCG  policies  call  for 

•  testing  MISLE  software  in  successively  larger  increments^ 

•  unit  testing 

•  functional  testing 

•  integration  testing 

•  system  testing 

•  operational  test  and  evaluation  (OT&E) 

•  managing  problems  reported  during  testing: 

•  problem  reports  are  to  be  prioritized  in  levels  1  to  5  (levels  1  and  2 
are  most  critical),  fixed,  tested,  and  closed 

•  all  critical  problem  reports  (levels  1  and  2)  are  to  be  closed  before  the 
next  stage  of  testing  begins 

2We  define  similar  concepts  in  our  guide  for  managing  testing  activities,  27 

Year  2000  Computing  Crisis:  A  Testing  Guide  (GAO/AIMD-1 0.1 .21 ,  November  1 998). 
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USCG’s  current  testing  schedule  for  MISLE  components: 


MSN  Integration 
Testing 

(4/1/01) 

(6/26/01) 

MSN  System 

(6/30/01) 

(8/27/01) 

Testing 

MSN  OT&E 

(9/10/01)  (9/21/01) 

VDS  Integration 

(5/21/01) 

(8/10/01) 

Testing 

VDS  System 
Testing 

(8/16/01)  (8/27/01) 

VDS  OT&E 

(9/10/01)  (9/21/01) 

April 

May 

June  July 

Aug  Sept 
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Key  Risk  Areas 

Software  Testing  (cont’d) 


USCG  has  undertaken  risky  testing  practices: 


•  Key  VDS  functions  were  not  tested  during  functional  testing  because 
they  were  under  development  at  the  time  and  were  deferred  to 
integration  testing.  For  example: 


•  system  security 

•  processing  and  printing  Abstracts  of  Titles 

•  printing  Certificates  of  Documentation 
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Risky  testing  practices  (cont’d): 

•  Significant  problems  were  not  closed  before  the  next  stage  of  testing 
began.  For  example: 

•  9  critical  problems  identified  during  MSN  integration  testing  were 
not  closed  before  system  testing  began 


USCG’s  testing  practices  increase  the  risk  that  MISLE  will  not  perform  as 
expected,  or  may  take  longer  to  develop  than  expected. 
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Key  Risk  Areas 

Transition  Planning 


According  to  the  CIO  Council’s  guidance  on  enterprise  architectures, 
agencies  should  carefully  plan  the  transition  from  legacy  systems  to  new 
systems.  To  ease  the  transition  from  MSIS  to  MISLE,  USCG  developed  a 
transition  plan  that  calls  for  developing  data  migration  plans  and  training 
system  users. 


USCG  has  acknowledged  that  transitioning  from  MSIS  to  MISLE  will  be 
difficult.  It  will  involve  moving  data  from  one  system  to  the  next,  ensuring 
the  accuracy  of  the  transported  data,  and  training  users  on  new  business 
processes.  USCG  has  undertaken  several  transition  efforts: 


•  MSN  data  migration  plan  has  been  developed 

•  Hundreds  of  MSN  users  have  undergone  training 

•  A  4-week  period,  known  as  MISLE  month,  is  planned  in  which  MSIS 
and  MISLE  will  be  run  concurrently  to  allow  MSN  users  to  become 
acquainted  with  the  new  system  and  new  work  processes 
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Key  Risk  Areas 

Transition  Planning  (cont’d) 


Despite  its  transition  guidance,  USCG  has  not  yet  completed  critical  VDS 
transition  planning  because  the  initial  system  deliverables  changed 
dramatically  in  July  2001 .  Specifically, 


•  VDS  transition  plan  has  not  been  finalized 

•  VDS  training  plan  has  not  been  developed 

•  VDS  users  have  not  been  trained 


VDS  users  stated  that  training  and  transition  planning  are  not  critical  for 
the  first  release  of  VDS,  because  the  functions  being  delivered  will  not 
dramatically  affect  their  operations.  However,  they  acknowledge  that 
transition  and  training  are  critical  to  the  next  software  release, ^  which  is 
planned  to  reengineer  and  automate  their  work  processes. 

Incomplete  plans  for  transitioning  to  a  new  system  increase  the  likelihood 
that  the  system  will  not  be  deployed  on  schedule.  Insufficient  training 
plans  also  increase  the  likelihood  that  users  will  not  readily  adapt  to  new 
work  processes  and  systems. 


^USCG  has  not  yet  scheduled  the  next  software  release  for  VDS. 
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Key  Risk  Areas 

Risk  Management 


Risk  management  is  a  key  component  of  a  sound  systems  development 
approach.  An  effective  risk  management  approach  typically  includes 
identifying,  prioritizing,  resolving,  and  monitoring  project  risks.  In  support  of 
this  approach,  USCG’s  risk  management  plan  calls  for 


•  assigning  a  severity  rating  (high,  medium,  or  low)  to  risks  that  bear 
particular  attention  and  placing  these  risks  on  a  “risk  watch  list,” 

•  prioritizing  these  risks, 

•  planning  a  response  or  strategy  for  each  risk  on  the  risk  watch  list,  and 
drafting  a  detailed  response  plan,  and 

•  reviewing  and  evaluating  all  risks  on  the  the  risk  watch  list  during 
monthly  management  meetings. 
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Key  Risk  Areas 

Risk  Management  (cont’d) 


USCG  has  not  effectively  implemented  its  risk  management  plan. 


•  USCG  developed  three  different  risk  lists,  and  none  of  the  risks  were 
assigned  a  severity  rating. 

•  Risks  on  two  of  the  risk  lists  have  not  been  prioritized. 

•  USCG  developed  detailed  mitigation  strategies  for  some,  but  not  all  risks. 
For  example, 

•  detailed  plans  exist  for  data  migration  and  transition  risks 

•  such  plans  do  not  exist  for  managing  user  expectations  and  VDS 
instability  risks 

•  USCG  officials  addressed  some,  but  not  all,  active  risks  at  monthly  status 
briefings.  Additionally,  USCG  officials  stated  that  they  discuss  all  active 
risks  during  monthly  management  meetings;  however,  we  found  no 
evidence  that  all  risks  were  addressed  at  these  informal  meetings. 


Inadequate  risk  management  practices  increase  the  likelihood  that  MISLE  will 
not  be  delivered  on  schedule,  within  budget,  and  able  to  perform  as  expected. 
USCG  officials  acknowledged  areas  where  they  could  improve  their  risk 
management  and  have  undertaken  improvement  efforts. 

34 


Page  37 


GAO-02-11  Coast  Guard 


Appendix  I 

GAO’s  September  18,  2001,  Briefing 


GAO 

Accountability  *  Integrity  *  Reliability 


Conclusions 


After  a  long  history  of  problems  in  developing  MISLE,  USCG  ended  the 
original  MISLE  contract  in  October  1999.  Since  that  time,  USCG  has  made 
progress  in  developing  MISLE  and  is  now  poised  to  deploy  a  minimum  level 
of  functionality  in  November  2001 .  However,  MISLE  is  several  months 
behind  schedule  and  USCG  has  already  spent  most  of  its  $61  million 
acquisition  cost  estimate-most  of  it  on  the  original  contract.  With  only  about 
$1 1  million  remaining,  much  remains  to  be  done. 

In  its  efforts  to  develop  and  deploy  a  complete  MISLE  system,  USCG  faces 
significant  challenges  in  several  areas: 

•  managing  system  requirements  and  user  expectations 

•  testing  the  system 

•  transitioning  to  an  operational  system,  and 

•  managing  program  risks 

Unless  USCG  can  effectively  address  these  challenges,  MISLE  deployment 
schedules  are  likely  to  slip  further  and  costs  are  likely  to  increase.  Further, 
MISLE  may  fall  short  of  user  expectations  and  its  promised  functionality. 
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Recommendations 


To  mitigate  MISLE  risks,  we  are  making  several  categories  of 
recommendations  to  USCG’s  MISLE  program  managers: 


System  Requirements 

•  Define  and  prioritize,  in  conjunction  with  system  users,  all  needed 
system  functions,  corrections,  and  enhancements  that  must  occur  to 
meet  valid  user  needs. 

•  Develop  cost  and  schedule  estimates  for  providing  these  functions, 
corrections,  and  enhancements. 


Software  Testing 

•  Close  all  critical  problems  before  initiating  the  next  stage  of  testing. 
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Recommendations  (cont’d) 


Transition  Planning 

•  Finalize  and  implement  VDS  transition  plans. 

•  Develop  and  implement  VDS  training  materials. 


Risk  Mitigation 

•  Develop  a  single  list  of  system  risks. 

•  Evaluate  system  risks  to  determine  their  severity  and  prioritize  these 
risks. 

•  Develop  and  implement  comprehensive  mitigation  strategies  for  each 
of  the  risks. 

•  Regularly  oversee  the  status  of  risks  and  risk  mitigation  efforts  to 
determine  whether  additional  mitigation  activities  are  warranted. 
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Agency  Comments  and  our  Evaluation 


In  providing  oral  comments  on  a  draft  of  this  briefing,  USCG  officials 
generally  agreed  with  our  recommendations,  but  disagreed  with  our 
conclusion  that  unless  the  agency  can  address  key  risk  areas,  MISLE 
deployment  schedules  are  likely  to  slip  further,  costs  are  likely  to  increase, 
and  MISLE  may  fall  short  of  user  expectations  and  its  promised 
functionality. 


USCG  officials  stated  that  many  of  the  changing  requirements  and  testing 
issues  we  raised  were  “tradeoff”  decisions  made  to  meet  the  primary  goal 
of  delivering  an  initial  version  of  MISLE  to  replace  MSIS.  Additionally, 
USCG  officials  stated  that  some  MISLE  functions  were  deferred  to  later 
releases  in  order  to  accelerate  other  user-requested  functions.  USCG 
officials  expressed  confidence  that  they  will  deliver  MISLE  within  budget 
and  on  schedule. 
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Agency  Comments  and  our  Evaluation 


Our  evaluation:  We  acknowledge  that  USCG  expects  to  meet  its  primary 
goal  of  replacing  MSIS  in  the  next  few  months.  However,  MISLE  will  now 
provide  fewer  capabilities  than  originally  planned,  and  significant  functions 
have  been  delayed  because  of  problems  in  testing. 


By  dropping  and  delaying  key  MISLE  functions,  USCG  runs  the  risk  that  the 
system  will  fall  short  of  user  needs.  Further,  because  there  are  no  schedule 
or  cost  estimates  for  most  of  the  delayed  functions,  it  is  not  clear  when  they 
will  be  delivered  or  what  they  will  cost. 
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The  General  Accounting  Office,  the  investigative  arm  of  Congress,  exists  to 
support  Congress  in  meeting  its  constitutional  responsibilities  and  to  help  improve 
the  performance  and  accountability  of  the  federal  government  for  the  American 
people.  GAO  examines  the  use  of  public  funds;  evaluates  federal  programs  and 
policies;  and  provides  analyses,  recommendations,  and  other  assistance  to  help 
Congress  make  informed  oversight,  policy,  and  funding  decisions.  GAO’s 
commitment  to  good  government  is  reflected  in  its  core  values  of  accountability, 
integrity,  and  reliability. 
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GAO  Reports  and 
Testimony 

The  fastest  and  easiest  way  to  obtain  copies  of  GAO  documents  is  through  the 
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electronic  order  form  found  under  “To  Order  GAO  Products.” 
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Washington,  D.C.  20013 

To  order  by  Phone:  Voice:  (202)  512-6000 

TDD:  (301)413-0006 

Fax:  (202)  258-4066 

Visit  GAO’s  Document 
Distribution  Center 

GAO  Building 

Room  1100,  700  4th  Street,  NW  (comer  of  4th  and  G  Streets,  NW) 

Washington,  D.C.  20013 

To  Report  Fraud, 
Waste,  and  Abuse  in 
Federal  Programs 

■ 

Contact: 

Web  site:  www.gao.gov/fraudnet/fraudnet.htm, 

E-mail:  fraudnet@gao.gov,  or 

1-800-424-5454  (automated  answering  system). 

■ - 

Plll^lic  AffrlirS  NelUgan,  Managing  Director,  NelliganJ@gao.gov  (202)  512-4800 

Kxu  K.  Cl  o  U.S.  General  Accounting  Office,  441 G.  Street  NW,  Room  7149, 

Washington,  D.C.  20548 


PRINTED  ON  (g  A  RECYCLED  PAPER 


United  States 

General  Accounting  Office 

Washington,  D.C.  20548-0001 

Official  Business 
Penalty  for  Private  Use  $300 


Presorted  Standard 
Postage  &  Fees  Paid 
GAO 

Permit  No.  GIOO 


Address  Correction  Requested 


